constrain-JULES-ES-1p0.Rmd
Doug McNeall
1/19/2021
Sensitivity analysis and constraint of the parameter space of Earth System configuration of JULES.
Update of explore-JULES-ES-1p0.Rmd that uses the new packages and functions
Andy thinks there might be mileage in analysing the atmospheric growth, which is here. /home/h01/hadaw/Research/ES_PPE/Oct20
At the moment, this vignette is hampered by the fact that emulators are failing on a few of the outputs which represent change over the historical perid. The emulator is fine for predicting absolute values in the modern period.
# Load helper functions
knitr::opts_chunk$set(fig.path = "figs/", echo = FALSE, message = FALSE, warnings = FALSE)
# load some helper functions
source('~/brazilCSSP/code/brazil_cssp/per_pft.R') # eventually, move the relevant functions
source('explore-JULES-ES-1p0_PPE_functions.R')Jules output data location
Extract a time series of an annual, globally aggregated variable
Test with the global sum of NPP and plot the timeseries
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0069/stats/JULES-ES-1p0_P0069_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0086/stats/JULES-ES-1p0_P0086_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0144/stats/JULES-ES-1p0_P0144_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0165/stats/JULES-ES-1p0_P0165_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0168/stats/JULES-ES-1p0_P0168_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0183/stats/JULES-ES-1p0_P0183_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0185/stats/JULES-ES-1p0_P0185_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0214/stats/JULES-ES-1p0_P0214_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0216/stats/JULES-ES-1p0_P0216_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0235/stats/JULES-ES-1p0_P0235_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0238/stats/JULES-ES-1p0_P0238_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0251/stats/JULES-ES-1p0_P0251_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0259/stats/JULES-ES-1p0_P0259_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0285/stats/JULES-ES-1p0_P0285_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0357/stats/JULES-ES-1p0_P0357_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0376/stats/JULES-ES-1p0_P0376_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0387/stats/JULES-ES-1p0_P0387_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0394/stats/JULES-ES-1p0_P0394_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0405/stats/JULES-ES-1p0_P0405_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0424/stats/JULES-ES-1p0_P0424_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0455/stats/JULES-ES-1p0_P0455_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
## Error in R_nc4_open: No such file or directory
## Error in nc_open(paste0(fn)) :
## Error in nc_open trying to open file /project/carbon_ppe/JULES-ES-1p0_PPE/P0457/stats/JULES-ES-1p0_P0457_Annual_global.nc
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C call
## Error in R_nc4_close: NetCDF: Not a valid ID
Function to extract the “modern value” direct from the file (last 20 years of the timeseries)
Loop to extract the “modern value” of a number of model outputs
Calculate an ensemble of anomalies for all variables
For each ensemble member and each variable, calculate the change from the 20 years at the start of the run, to the twenty years at the end of the run.
## [1] 2.410932e-09Probability of run failure
Could be an interesting project - logit transformation for probability of run failure? Some other ML like SVM?
Clean data sets
## [1] TRUEWhere did the model fail to run?
There are clear run failure thresholds in the parameters rootd_ft_io and lai_max, and quite strong visual indications that a_wl_io and bio_hum_cn matter.
Build some Gaussian Process emulators
Andy would like to see timeseries of: cVeg, cSoil and nbp, npp in GtC and GtC/yr.
First, use mean NPP as an example. How does NPP respond to each parameter? NAs are removed, but zero values are still included.
Relationship between modern value and change since 1850.
Some outputs (e.g fLuc, fHarvest) have an almost perfect 1:1 relationship between modern value and change, some (nbp, npp, treeFrac) quite or moderately strong, and some (csoil, cveg) very weak or non-existant. 
A clear threshold in the F0 parameter.
It appears that this ensemble is less “clear cut” in having an output that clearly distinguishes between “failed” (or close to it), and “not failed”.
Having said that, having an F0 over a threshold seems to kill the carbon cycle, as before. Here, we’ve set a threshold of 0.9 (on the normalised scale) for F0, and we remove members of the ensemble with a larger F0 than that when we build emulators.
Remove anything with f0 over a threshold (level 1 constraint) .
The level 1 constraint removes any input with F0 greater than 0.9 (normalised), which removes many of the zero-carbon-cycle members up front. There are 424 ensmble members remaining.
This does constraint sequentially, which may not be a good idea.
# Plot the regular km emulator. Doesn’t look great.
Constrain first and then do sensitivity analysis?
The problem with doing constraint first is that you end up with a non-hypercube shaped input space (corners have been knocked off by constraint), which is not ideal. We might therefore want different sensitivity measures for pre- and post-constrained ensemble.
Initial sensitivity Matrix
It looks as though bwl_io is very influential across a number of variables, even though it didn’t appear that interesting in the parginal plots. Why is that?
Re-examine bwl_io
A closer look at bwl_io now that the impact of f0_io has been removed shows a large number of “zero” NPP when bwl_io is at low values, which could well be the source of apparent sensitivity.
Take only higher values of bwl_io for the next round of constraints.
Plot both sensitivity matrices on top of one another.
It looks as though both the absolute value and the change over time are controlled by the same parameters. 
How good are the emulators for each output?
The emulators appear to be at least capturing the broad response for all of the output variables.
First, plot the straight kriging emulators
km emulators for change in variables over time
Next, plot the twostep glmnet/km emulators
TwoStep emulator performance for modern values
Direct twoStep emulator using foreach
History matching and generation of new ensemble members
Augment the design.
The function addNroyDesignPoints builds an emulator for each model output in Y. It compares the output of each emulator at a number of candidate desin points, and chooses a space-filling set of them that that are Not Ruled Out Yet (statistically close to the observation at Y_target).
Write the augmented design
The function write_jules_design here simply takes the points calculated by addNroyDesignPoints and writes them to configuration files.
Check the design
Check that the augmented design produces what we expect. New ensemble members should be somewhat constrained within the boundaries of the original design, if the comparison to data offers any constraint.
